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<> The Testing of Proficiency 
* in Listening Comprehension: 
Guidelines for Teaching 



p Charles J. James 

University of Wisconsin-Madison 



Since 1982 the American Council on the Teaching of Foreign Languages 
has been promoting discussion and use of its Provisional Proficiency Guide- 
lines.' The history of the Guidelines is presented extremely well by Liskin- 
Gasparro,' with an update by Lowe.' The greatest attention to date has 
been focused on the speaking skill through the medium of the "Oral 
Proficiency Interview." The other areas (reading, writing, listening, and 
culture) have yet to enjoy the same amount of attention and research 
activity, although there are healthy signs that this will change in the near 
future, at least for listening and reading. 4 

The purpose of this paper is to outline the guidelines for listening 
comprehension, and to make suggestions for the testing and teaching of 
listening comprehension skills. 

Listening seems to be the most neglected of language skills, according 
to Chastain and others. 1 Ironically, it is the most immediate skill used in 
most modern language classes, since the development of speaking skills 
usually requires that students listen to substantial amounts of material 
before they can repeat, answer questions, or form original utterances. 
Even reading makes use of listening skills, especially in beginning courses. 
However, listening is itself a significant skill area, which can be dealt with 
separately.* The major premise here is that listening is not only the first 
skill in the traditional sequence of listening-speaking-reading-writing, but 
is also the most basic skill to be developed before a learner can become 
V) genuinely independent in the language, whatever skills may eventually be 
ro needed for continued study or professional advancement. 

■J 
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The rest of this article deab with five questions that seem to be basic 
to developmg proficiency in listening comprehension. 

i. What is the nature of the relationship between listening and speak- 

In general, listening comprehension ability will not only nrecede 
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prehensron should be generally more advanced than speaking at all but 
the very lowest levels of speaking proficiency. 

2. What kinds of listening tasks should be encouraged in the class- 
room-which by means of taped or filmed material, and which by 
means of direct contact with native speakers? 
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3. How much "editing" of listening materials should be done, espe- 
cially at the lower levels of proficiency? 
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the teacher or the examiner wishes to emphasize. Any spoken can ' 
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be used for just about any purpose, as long as students are given realistic 
tasks to perform in connection with their listening. Listening to native 
speakers converse on their own turf at their own speed can be a thoroughly 
traumatic experience without the buffer of good listening comprehension 
tasks. For most listening comprehension material commonly in use in the 
United States, especially in the form of language laboratory tapes produced 
for first-year textbooks, the problem is not the level of content difficulty, 
but rather the lack of interest generated in the learner. 10 The issue is not * 
"editing** in terms of simplification, but a matter of selecting appropriate 
and motivating material that yields successful learning tasks. 

4. Are the guidelines for listening comprehension proficiency 
sufficiently developed to yield "ratable" samples of a learner s ■ 
listening ability? 

The ACTFL Provisional Proficiency Guidelines for listening are di- j 
vided into four basic levels: Novice, Intermediate, Advanced, and Superi- 
or. The Novice and Intermediate levels are further divided into three 
sublevels: Low, Mid, and High. The Advanced level is divided into Ad- 
vanced and Advanced Plus. Underlying the four levels and their sublevels 
is the so-called "Functional Trisection," which describes the criteria for 
judging the overall "quality" of a given performance sample. The Trisec- 
tion divides a sample into "Functions," that is, the language tasks that a 
person is expected to perform; "Content/Context," the topics and situa- 
tions in which a person would be expected to operate; and "Accuracy," 
the degree of precision with which a given message is to be received and 
acted upon by the learner or candidate. 

The following tables present brief descriptions of the Guidelines for 
listening, across the four major levels and the three aspects of the Func- 
tional Trisection. j 

FUNCTIONS 

Superior Distinguishes points of view 
Follows negotiations 
Appreciates persuasion 
Understands counseling 
Responds to language tailored to audience 
Operates in unfamiliar situations 
Understands supported opinion' 
Reconstructs hypotheses 
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Advanced Operates in survival situations involving complications 
Follows narration 
Processes description 

Intermediate Operates in simple survival situations 
Understands questions 

Responds positively (appropriately) to creative language 
Novi.Te Understands memorized material 

CO^XT/CONTENT 

Superior AH nontechnical situations 

Any conversation within the range of personal or professional 
experiences 

All topics normally pertinent to professional needs and social needs 

of a general nature 
Expression and defense of opinions about current events 
Most formal and informal conversations 
Practical, social professional and abstract topics 

Advanced Particular interests and special fields of competence of the speaker 
(listener) 
Recreational activities 
Limited work requirements 
Most social situations, including introductions 
Concrete topics, such as work, travel, background, etc. 

Intermediate Simple question-and-answer situations 

Familiar topics within scope of limited Language experience 

Routine travel needs 

Minimum courtesy requirements 

Everyday survival topics 

Novice Basic objects Weekdays Colors 
Months Clothing Day $ date 
Family Time Weather 

ACCURACY 

Superior Only occasional unpattemed comprehension errors 
Nearly perfect grammatical comprehension 
Occasional misunderstanding in low-frequency structures 
Occasional misunderstanding in most complex frequent structures 
Sporadic misunderstanding in basic structures 

Advanced Comprehension of compound sentences in limited discourse 
Good control of morpho-syntis of the language 
Very accurate understanding of elementary structures 
Understanding of a native speaker not used to dealing with 
foreigners 

Intermediate Misunderstanding even in constructions which are simple and 
common 

Understanding of a native speaker used to dealing with foreigners 
Novice Little or no functional comprehension 

Canale et al. present a similar table, considerably simplified, which 
combines listening and its printed partner, reading. 11 In addition, there is 
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several comprehension difficulties, since the interaction between two or 
more speakerscan either improve or complicate the comprehension process 
A listening comprehension test should be made up primarily of mono- 
logues and d.alogues. with monologues presented first. Three or four short 
monologues, each with a different content/context, should be followed by 
two or three longer dialogues. 

Song is a special category of listening material. The term refers here 
to any kind of listening enhanced by non-verbal sound, typically music 
(as m a lyric song), but also including chant, poetry reading, sermonizing 
public address announcements, and most broadcast commercials, especial- 
ly where a conscious effort is made to exaggerate the spoken message for 
emotional or financial reasons. A listening comprehension proficiency test 
should contain at least two or three such items, according to the level 
Here are some suggestions for these four types of listening material: 
Dictations can be constructed to suit any proficiency level. At the 
Advanced and Superior levels, real, uncut listening texts should be used 
for transcription, since the listener will be operating almost exclusively 
within the target language milieu. 

Examples of monologues can be found in the language normally 
presented on, the radio. For the Novice level the candidate could be 
expected to understand station identification, short introductions to pro- 
T^A J^ nC /: hecks - and the I"*- At the Intermediate level the candidate 
should be able to comprehend weather reports, traffic reports, program 
previews, and brief public service announcements. At the Advanced level 
entire news broadcasts, radio plays, music and special events programs 
could be presented for comprehension activities. At the Superior level the 
candidate should comprehend editorials, special news and information 
programs, comedy, and debates. 

Broadcast commercials present interesting challenges. Some work 
well at lower levels because of familiarity with the product or the relative 
hmness of the text. Others, even if they contain otherwise comprehensi- 
b e structure and vocabulary, may be difficult to process because of speed 
of delivery or compression of speech patterns. But commercials are usually 
good examples of what I have defined here as song, and have the added 
advantage of being very short. It should be possible to use seven or eight 
of them ,n a test battery with little processing difficulty on the part of the 
candidate. They are also frequently quite motivating for students 
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The basic testing procedure for listening comprehension proficiency 
would provide for a 20-30 minute tape of materials designed for testing 
at a given level. The tape would be accompanied by a worksheet which 
could be administered in writing (or orally if an examiner is to be present). 
At the Novice levels some physical activity, such as standing up, opening 
a window, etc., could be performed from the taped stimuli, or drawings 
or pictures could be presented for the candidate to identify on the basis 
of the recorded text." 

There are good administrative arguments for giving a multiple- 
choice test, such as those used in commercially available test batteries. 
However, more effective demonstration of comprehension could be elic- 
ited by having candidates respond to questions about the spoken test in 
their native language (at the Intermediate levels) or in the target language 
(at the Advanced Plus and Superior levels), to the extent that problems of 
reading comprehension did not interfere. If an examiner is to be present, 
it is possible for him or her to engage the candidate in a conversation, in 
the native or target language, about the content of the taped material. 

A most promising technique for eliciting listening comprehension 
proficiency without prejudicing the outcome or contaminating the data, 
is the "immediate recall protocol," as suggested by Bernhardt for read- 
ing." In preliminary research the protocol seems to be extremely effective 
for determining what students retain from the listening (or reading) expe- 
rience, as long as they are allowed to give whatever information or reac- 
tions, in whichever language they wish. This immediately raises the question 
of test reliability, since it is not always possible to determine just exactly 
what information or reactions will or should be elicited from a spoken text 
passage, or, indeed, how to evaluate accurately what is produced. 

Rating a sample of listening comprehension is still a matter of debate. 
If multiple-choice test items are used, a score that can be calibrated and 
statistically analyzed is possible. Better yet would be an examiner's judg- 
ment of the "functional success" of the candidate in exhausting his or her 
understanding of the material. Research needs to be done on the kinds of 
responses generated within the "immediate recall protocol" in order to 
determine which aspects of the Functional Trisection are most promising 
with regard to a long-term assignment of a level rating within the Proficiency 
Guidelines. 

One last thought: as indicated earlier, it should be possible to use just 
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about any listening material for just about any teaching or testing purpose. 
The assumption all along has been that there are individual texts which 
are better for one level of proficiency than for another. However, it could 
be that one text may be useful for all levels of proficiency simultaneously, 
recognizing that different levels are defined in terms of what individual 
human beings do with a given language sample. In other words, the 
problems may not be to search through language material for a group of 
texts that are appropriate for Novice or Superior level candidates, but 
rather for texts that can elicit Novice, Intermediate, Advanced, and Supe- 
rior level tasks. The division into monologues, dialogues, and songs (plus 
or minus dictation) seems to be a useful first step in producing a test 
battery for listening comprehension. 
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